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THE  ARMY  FAMILY  RESEARCH  PROGRAM  AND  THE  SELECTION 
AND  CLASSIFICATION  PROJECT  DATA  BASE 

INTRODUCTION 

In  1982,  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences  (ARI)  initiated  Project  A,  Improving  the  Selection,  Classification, 
and  Utilization  of  Army  Enlisted  Personnel.  Project  A  is  one  of,  if  not  the, 
most  comprehensive  and  long-range  research  projects  ever  undertaken  in  the 
social  sciences.  The  overall  goal  of  the  project  is  the  development  of  a 
computerized  personnel  allocation  system  that  will  match  available  personnel 
resources  with  Army  manpower  requirements,  taking  into  consideration  the 
aptitudes,  interests,  backgrounds,  and  performance  levels  of  first  and 
second  tour  soldiers. 

In  1986,  ARI  initiated  another  large,  long-range  project,  the  Army 
Family  Research  Program  (AFRP).  The  overall  goal  of  this  research  is  to 
establish,  on  the  basis  of  empirical  evidence  and  working  models,  the 
relationships  between  family  factors  and  Army  community  and  social  programs 
on  the  one  hand,  and  soldier  retention  and  readiness  on  the  other.  Both 
Project  A  and  the  Family  Research  Program  are  being  conducted  by  the 
Manpower  and  Personnel  Research  Laboratory  (MPRL)  at  ARI. 

At  the  kick-off  meeting  for  the  Family  Research  Program  on  December  16, 
1986,  Dr.  Kent  Eaton,  the  MPRL  Director,  urged  the  participants  to  take 
maximal  advantage  of  the  Project  A  data  base  in  conducting  AFRP  research. 
The  operational  objectives,  research  designs,  and  schedules  of  both  projects 
do,  in  fact,  permit  AFRP  researchers  to  capitalize  on  the  work  of  Project  A. 
In  terms  of  objectives,  both  projects  plan  to  use  job  performance  and 
retention  as  basic  criteria.  Both  projects  look  at  performance  indexes  as 
predictors  as  well:  In  Project  A,  earlier  performance  (e.g.,  at  Advanced 
Individual  Training)  will  be  used  to  predict  later  on-the-job  performance; 
in  the  AFRP,  models  could  capture  the  apparent  impact  of  earlier 
performance/readiness  on  later  performance/readiness. 
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Both  projects  are  also  concerned  with  individual  differences.  The 
measurement  of  individual  differences  in  terms  of  aptitudes,  interests,  and 
non-cognitive  predictors  is  a  major  focus  of  Project  A  selection  research. 
While  the  emphasis  on  individual  differences  in  the  AFRP  is  less,  they 
cannot  be  ignored  by  AFRP  researchers.  The  vast  literature  showing  the 
relationships  between  job  performance  criteria  and  retention  and  measures  of 
individual  differences  would  attest  to  the  incompleteness  of  any  AFRP  model 
relating  family  factors  to  soldier  performance,  readiness,  or  retention  that 
did  not  also  include  measures  of  individual  differences  as  antecedent, 
explanatory  variables. 

Though  Project  A  was  initiated  four  years  earlier  than  the  Family 
Research  Program,  the  current  schedules  of  both  projects  are  sufficiently 
congruent  to  allow  the  administration  of  AFRP  questionnaires  to  samples  of 
soldiers  participating  in  Project  A  research.  There  are  five  points  at 
which  this  overlap  in  sampling  might  take  place.  The  AFRP  has  already  taken 
advantage  of  the  first  one  at  Advanced  Individual  Training  (AIT).  Project  A 
is  currently  in  the  process  of  collecting  predictor  and  school  criteria  from 
the  46,500  first  tour  soldiers  in  21  Military  Occupational  Specialties  (MOS) 
that  constitute  the  Project  A  longitudinal  validation  sample.  At  the 
request  of  the  AFRP,  Project  A  staff  are  administering  an  AFRP  questionnaire 
to  about  10,000  of  these  soldiers  immediately  before  their  graduation  from 
AIT. 


Approximately  12,000  soldiers  of  the  Project  A  longitudinal  validation 
sample  will  be  administered  a  series  of  performance  measures  in  the 
summer/fall  of  1988.  At  the  same  time,  performance  mea.ures  vi  1 1  be 
administered  to  about  1,000  soldiers  in  their  second  tour  of  duty  who  were 
part  of  the  Project  A  concurrent  validation  sample.  These  soldiers  (see 
Figure  1)  were  administered  the  Project  A  battery  of  predictor  tests  and 
performance  measures  during  their  first  tour  in  the  summer/fall  of  1985. 
There  is  an  opportunity  to  distribute  (and  possibly  administer)  an  AFRP 
questionnaire  to  these  soldiers  immediately  after  they  have  finished  taking 
the  Project  A  performance  measures.  During  the  same  sessions,  soldiers  who 
are  married  could  be  requested  to  place  their  spouses'  addresses  on 
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Figure  1.  The  overall  data  collection  plan  for  Project  A 


envelopes  that  would  be  used  to  send  AFRP  questionnaires  to  the  spouses. 
The  responses  of  the  Project  A  soldiers  and  their  spouses  could  be  related 
to  the  predictor  and  performance  variables  in  the  Project  A  data  base. 

The  third  and  fourth  opportunities  to  capitalize  on  the  Project  A  data 
base  will  arise  during  the  major  AFRP  data  collections  (assuming  there  will 
be  two  waves  of  AFRP  data  collection).  By  the  time  these  data  are 
collected,  Project  A  will  have  administered  a  battery  of  experimental 
predictor  tests  to  about  55,000  enlisted  soldiers.  Project  A  performance 
measures  will  probably  have  been  administered  to  over  20,000  of  these 
soldiers.  As  the  AFRP  sampling  plan  has  not  been  finalized  yet,  it  is  not 
possible  to  predict  at  this  time  how  many  of  these  soldiers  will  also  enter 
into  the  AFRP  samples.  There  is  a  good  chance,  however,  that  enough 
soldiers  will  be  in  samples  of  both  projects  to  provide  very  useful 
additional  predictor  and  performance  data  for  the  AFRP  modeling  efforts. 
This  would  be  especially  true  if  a  deliberate  attempt  were  made  to  increase 
the  probability  of  including  Project  A  tested  soldiers  in  the  AFRP  samples, 
especially  those  who  completed  the  AFRP  AIT  Graduate  Survey. 

The  fifth  point  at  which  the  AFRP  could  take  advantage  of  Project  A 
data  collections  is  during  the  planned  administration  of  performance 
measures  in  1991  to  about  1,000  second  tour  soldiers  of  the  Project  A 
longitudinal  validation  sample.  As  indicated  for  the  first  tour  soldiers, 
an  AFRP  questionnaire  could  be  distributed  or  administered  during  the  same 
sessions  that  the  performance  measures  are  administered. 

It  should  be  kept  in  mind  that  starting  with  the  AFRP  AIT  survey,  there 
is  the  possibility  that  some  soldiers  could  have  AFRP  questionnaires 
administered  as  many  as  five  times  over  the  scheduled  five-yeai  span  of  the 
AFRP.  Though  the  possibility  of  obtaining  a  complete  set  of  Project  A/AFRP 
data  is  admittedly  slim  for  any  one  soldier,  considering  the  thousands  of 
soldiers  involved  in  the  Project  A  and  AFRP  samples,  the  number  of  soldiers 
for  which  there  would  be  complete  or  fairly  complete  data  may  allow  some 
special  longitudinal  analyses  to  be  performed.  This  would  be  especially 
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true,  if,  as  mentioned  earlier,  it  were  deemed  appropriate  to  deliberately 
attempt  to  increase  the  overlap  between  the  Project  A  and  AFRP  samples. 

Purpose 

The  purpose  of  this  working  paper  is  not  to  promote  any  specific  sampling 
plan  for  the  AFRP.  Instead,  it  is  intended  to  achieve  two  limited  objectives: 

(1)  To  acquaint  AFRP  researchers  with  the  scope  and  current  plans  of 
Project  A,  so  that  they  can  better  understand  hew  the  Project  A  data  base 
could  and  could  not  be  used  to  the  advantage  of  the  AFRP;  and 

(2)  To  describe  the  kinds  of  analyses  that  are  planned  for  the  first 
Project  A/AFRP  joint  data  base  that  is  composed  of  the  AFRP  AIT  Graduate 
Survey,  and  the  Project  A  predictor  and  school  performance  measures. 

It  is  worth  emphasizing  at  this  point  that  the  relevant  portions  of  the 
Project  A  data  base  would  be  provided  to  AFRP  researchers  through  the  good 
offices  of  MERL  and  Project  A  researchers.  No  attempt  must  be  made  to  upstage 
these  researchers  in  the  analysis  of  the  data  base  they  have  so  painstakingly 
pot  together.  Analyses  involving  Project  A  data  conducted  by  AFRP  researchers 
should  be  directed  solely  at  filling  gaps  in  the  data  collected  by  the  AFRP, 
which  otherwise  would  leave  the  attempt  to  model  the  relationships  between 
family  factors  and  programs,  and  retention,  performance  and  readiness 
insufficient. 

The  next  section  of  this  working  paper  gives  a  more  complete  description 
of  Project  A.  For  the  most  part,  the  write-up  is  taken  verbatim  from  Project 
A  publications.  The  final  section  of  this  report  presents  the  first  of  a 
projected  series  of  plans  for  the  analyses  of  the  data  obtained  from  joint 
AFRP/Project  A  samples. 
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OVERVIEW  OF  PROJECT  A1 


Project  A  is  perhaps  the  largest  personnel  research  and  development 
project  ever  undertaken.  Its  general  purpose  is  to  develop  an  improved 
selection/classification  system  for  all  entry-level  positions  in  an 
organization  that  annually  recruits  400-500  thousand  people,  selects  100-120 
thousand  of  them,  and  assigns  each  individual  to  one  of  more  than  250  jobs. 
The  full  design  for  Project  A  covers  a  span  of  9  years.  It  is  now  in  its 
sixth  year. 

A  parallel  effort  to  Project  A  is  Project  B  (Development  of  a 
Computerized  Allocation  System),  which  has  the  responsibility  for  modeling 
the  labor  supply  and  labor  demand  components  of  a  fully  functioning 
personnel  allocation  system  and  for  developing  the  computer  algorithms  and 
software  to  integrate  supply  information,  demand  information,  and 
classification  validity  information. 

If  both  Projects  A  and  B  are  successful,  the  final  product  will  be 
composed  of  the  following  elements: 

t  A  labor  supply  forecasting  model  and  procedures  for  estimating 
the  parameter  values  of  the  model. 

t  A  model  for  forecasting  the  Army's  long-  and  near-term  personnel 
needs  (labor  demand)  and  procedures  for  estimating  the  parameter 
values  of  the  model. 

•  A  new  set  of  selection/classification  tests  which,  together  with 
the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB),  optimize 
the  balance  between  the  costs  of  testing  and  the  gain  in 
classification  utility. 

•  A  metric  and  procedure  for  estimating  the  utility  of  performance 
within  and  across  jobs. 


*  Sect  ions  of  this  Project  A  overview  were  excerpted  from  the  ARI 
Technical  Report  746  Improving  the  Selection.  Classification,  and 
Utilization  of  Army  Enlisted  Personnel:  Annual  Report,  1985  Fiscal  Year  by 
John  P.  Campbell,  Editor.  Alexandria,  VA:  U.S.  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences. 
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t  A  set  of  computerized  algorithms  (e.g.,  linear  programming)  that 
integrates  demand  information,  supply  information,  and  validity 
information  such  that  for  any  designated  period  the  overall 
utility  of  personnel  assignments  is  maximized. 

Specific  Objectives  of  Project  A 

The  project  has  two  principal  kinds  of  objectives.  The  first  type 
pertains  to  the  operational  needs  of  tne  Army.  They  constitute  the  basic 
purposes  for  which  the  project  is  funded  and  supported.  Specifically, 
Project  A  is  to: 

1.  Develop  new  measures  of  job  performance  that  can  be  used  as 
criteria  against  which  to  validate  selection/classification 
measures.  The  new  criterion  measures  will  use  a  variety  of  methods 
to  assess  both  job  specific  measures  of  task  performance  and 
general  performance  factors  that  are  not  job  specific. 

2.  Validate  existing  selection  measures  against  both  existing  and 
project-developed  criteria. 

3.  Develop  and  validate  new  and/or  improved  selection  and 

classification  measures. 

4.  Validate  proximal  criteria,  such  as  performance  in  training,  as 
predictors  of  later  criteria,  such  as  job  performance  ratings,  so 
that  more  informed  decisions  about  reassignment  and  promotion  can 
be  made  throughout  the  individual's  tour. 

5.  Determine  the  relative  utility  to  the  Army  of  different  performance 
levels  across  MOS. 

6.  Estimate  the  relative  effectiveness  of  alternative  selection  and 
classification  procedures  in  terms  of  their  validity  and  utility 
for  making  operational  selection  and  classification  decisions. 
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A  second  set  of  objectives  has  to  do  with  questions  of  a  more 
scientific  nature.  This  second  set  of  questions  is  being  addressed  with 
essentially  the  same  data  as  the  first.  That  is,  the  project  does  not  have 
two  parts  with  one  having  to  do  with  basic  research  and  the  other  focused  on 
applied  research.  Instead,  the  scope  of  the  project  and  the  attempt  to 
consider  an  entire  system  at  one  time  make  it  possible  to  concurrently 
address  a  number  of  more  basic  research  objectives.  Some  of  these  are  as 
follows: 

1.  Identify  the  basic  variables  (constructs)  that  constitute  the 
universe  of  information  available  for  selection/classification  into 
'■ntry- level  skilled  jobs. 

2.  Develop  a  comprehensive  model  of  performance  for  entry-level 
skilled  jobs  that  incorporates  both  a  theoretical  latent  structure 
and  linkages  to  state-of-the-art  measurement. 

3.  Describe  the  utility  functions  and  the  utility  metrics  that 
individuals  actually  use  when  estimating  “utility  of  performance." 

4.  Describe  the  degree  of  differential  prediction  across  (a)  major 
domains  of  abilities,  personality,  interests,  and  personal  history, 
(b)  major  factors  of  job  performance,  and  (c)  different  types  of 
jobs.  The  project  will  collect  a  large  sample  of  information  from 
each  of  these  three  populations  (i.e.,  individual  differences, 
performance  factors,  and  jobs). 

5.  Determine  the  extent  of  differential  prediction  across  racial  and 
gender  groups  for  a  systematic  sample  of  individual  differences, 
performance  factors,  and  jobs. 

6.  Develop  new  statistical  estimators  of  classification  efficiency. 
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Research  Design 


To  meet  these  objectives,  a  design  was  developed  that  uses  two 
predictive  and  one  concurrent  validation  on  two  major  troop  cohorts  (FY83/84 
accessions  and  FY86/87  accessions),  and  one  file  data  validation  on  the 
FY81/82  cohort.  That  is,  in  addition  to  collecting  data  from  new  samples, 
the  project  is  making  use  of  existing  file  data  that  have  been,  or  can  be, 
accumulated  for  1981  and  1982  accessions.  A  schematic  of  the  data 
collection  plan  was  shown  in  Figure  1  (page  3). 

The  logic  of  the  design  is  straightforward.  Existing  file  data  on  the 
FY81/82  cohort  provided  the  first  opportunity  to  revalidate  the  Armed 
Services  Vocational  Aptitude  Battery  (ASVAB)  against  existing  training 
criteria  and  against  the  Skill  Qualification  Test  (SQT).  The  results  of  the 
analyses  of  FY81/82  file  data  were  used  to  suggest  operational  changes  in 
ASVAB  composites.  The  file  sample  consisted  of  approximately  90,000  records 
distributed  over  120  M0S  in  sufficient  number  to  permit  analysis.  The 
FY81/82  data  also  provide  a  benchmark  against  which  to  compare  the 
additional  validation  data  to  be  collected. 

The  FY83/84  cohort  provided  the  first  opportunity  to  obtain  validation 
data  using  new  predictor  tests  and  new  job  performance  measures.  Two 
samples  have  been  taken  from  this  cohort.  First,  a  “preliminary"  predictor 
battery  of  predominantly  off-the-shelf  tests  chosen  to  represent  major 
constructs  was  administered  to  soldiers  in  four  M0S  (31C,  19E/K,  63B,  71L) 
as  they  entered  the  Army  during  the  last  half  of  FY83  and  the  first  half  of 
FY84.  A  total  of  11,000  personnel  in  the  four  M0S  were  tested.  Besides 
looking  at  the  relationship  of  the  Preliminary  Battery  constructs  to  the 
existing  ASVAB,  a  portion  of  this  sample  was  followed  during  the  summer  and 
fall  of  1985  with  a  broad  array  of  criterion  measures  (to  be  described). 
The  follow-up  of  the  Preliminary  Battery  sample  was  part  of  a  much  larger 
concurrent  validation  sample  drawn  from  1985  job  incumbents  who  entered  the 
Army  during  FY83/84. 
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Results  from  the  administration  of  the  preliminary  predictor  battery 
were  used  to  help  develop  the  Trial  predictor  battery  for  use  in  the  major 
concurrent  validation  during  the  summer  and  fall  of  1985.  Immediately  prior 
to  the  major  concurrent  validation  of  the  Trial  Battery,  all  predictors  and 
all  criterion  measures  were  put  through  a  series  of  field  tests.  For 
example,  all  criterion  measures  were  field  tested  on  approximately  150 
incumbents  in  each  of  nine  MOS.  The  test  battery  used  during  the  predictor 
field  tests  was  labeled  the  Pilot  Trial  Battery.  Both  the  Preliminary 
Battery  sample  and  the  field  tests  were  used  to  develop  the  Trial  Battery. 

The  Trial  Battery  was  validated  on  a  sample  of  approximately  9,000 
first  tour  enlisted  soldiers  in  19  MOS  against  an  array  of  newly  developed 
training  and  job  performance  measures.  As  noted  above,  a  subset  of  the 
concurrent  validation  sample  took  the  Preliminary  Battery  approximately  18 
months  earlier,  which  permitted  a  longitudinal  validation  of  the  off-the- 
shelf  tests  that  were  selected  to  represent  major  ability  and  personality 
constructs. 

Analysis  of  the  Trial  Battery  data  resulted  in  further  revision  of  the 
predictor  battery.  The  revised  version,  called  the  Experimental  Battery,  is 
being  used  with  a  longitudinal  validation  sample  selected  from  people  who 
entered  the  Army  in  FY86  and  FY87.  With  breaks,  the  battery  takes  four 
hours  to  administer.  The  Experimental  Battery  was  administered  at  the  time 
of  entry  to  approximately  46,500  people  distributed  across  21  MOS  (see  Table 
1).  The  training  measures  will  be  administered  at  the  conclusion  of  each 
individual's  Advanced  Individual  Training  (AIT)  course  and  the  job 
performance  criterion  data  will  be  collected  approximately  18  months  later, 
in  the  summer/fall  of  1988.  In  addition,  both  general  (Army-wide)  and  job 
(MOS)-specific  performance  measures  will  be  developed  and  administered  to 
the  surviving  members  of  both  the  FY83/84  and  FY86/87  cohort  samples  during 
their  second  tour  of  duty.  Consequently,  for  both  these  samples  the  design 
is  also  a  longitudinal  one. 
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Table  1.  Estimated  Number  of  Cases  in  Project  A  Longitudinal  Validation 
and  AFRP  AIT  Samples 


Military  Occuoational  SDecialtv 

Proiect  A 

Familv  Survey 

Infantryman 

11B 

13,040 

1,600 

Combat  Engineer 

12B 

2,240 

445 

Cannon  Crewman 

13B 

5,200 

1,350 

MANPADS  Crewman 

16S 

710 

180 

M48-M60  Armor  Crewman/ 

Ml  ABRAMS  Armor  Crewman 

19E/K* 

2,600 

460 

TOW/ DRAGON  Repairer 

27E 

90 

60 

Conmunications-Electronic 

Radio  Repairer 

29E 

260 

135 

Single  Channel  Radio  Operator 

31C 

770 

130 

Carpentry /Masonry  Specialist 

51B 

440 

85 

NBC  Specialist 

54B/C/E 

860 

425 

Ammunition  Specialist 

55B 

420 

130 

Light  Wheel  Vehicle  Mechanic 

63B 

1,420 

185 

Motor  Transport  Operator 

64C/88M 

1,840 

165 

Utility  Helicopter  Repairer 

67N 

280 

130 

Administrative  Specialist 

71L 

2,200 

385 

Unit  Supply  Specialist 

76Y 

2,430 

525 

Medical  Specialist 

91A 

3,990 

1,250 

Food  Service  Specialist 

94B 

3,430 

1,285 

Military  Police 

95B 

3,970 

950 

Intelligence  Analyst 

96B 

340 

120 

Totals 

46,530 

9,995 

*  19E  and  19K  are  separate  MOS. 
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Sample  Selection 


The  overall  objective  in  generating  the  samples  has  been  to  maximize 
the  validity  and  reliability  of  the  information  to  be  gathered,  while  at  the 
same  time  minimizing  the  time  and  costs  involved.  In  part,  costs  are  a 
function  of  the  numbers  of  people  in  the  sample.  However,  costs  are  also 
influenced  by  the  relative  difficulty  involved  in  locating  and  assembling 
the  people  in  a  particular  sample,  by  the  degree  to  which  the  unit's 
operations  are  disrupted  by  the  data  collection,  by  the  staff  costs  involved 
in  collecting  the  data  in  a  particular  manner,  and  by  other  such 
considerations. 

The  sampling  plan  itself  incorporated  two  principal  considerations. 
First,  a  sample  of  MOS  was  selected  from  the  universe  of  possible  MOS;  then, 
the  required  sample  sizes  of  enlisted  personnel  within  each  MOS  were 
specified.  The  MOS  are  the  primary  sampling  units.  This  design  is 
necessary  because  Project  A  is  developing  a  system  for  a  population  of  jobs 
(MOS),  but  only  a  sample  of  MOS  can  be  studied. 

Large  and  representative  samples  of  enlisted  personnel  within  each 
selected  MOS  are  important  because  stable  statistical  results  must  be 
obtained  for  each  MOS.  There  is  a  trade-off  in  the  allocation  of  project 
resources  between  the  number  of  MOS  researched  and  the  number  of  subjects 
tested  within  each  MOS:  The  more  MOS  are  investigated,  the  fewer  subjects 
per  MOS  can  be  tested,  and  vice  versa.  Cost  versus  statistical  reliability 
considerations  dictated  that  only  about  20  MOS  could  be  studied. 

The  selection  of  the  sample  of  MOS  proceeded  through  a  series  of 
stages.  An  initial  sample  of  MOS  was  drawn  by  using  the  following 
considerations: 

t  High  density  MOS  that  would  provide  sufficient  sample  sizes  for 
statistically  reliable  estimates  of  new  predictor  validity  and 
differential  validity  across  racial  and  gender  groups. 
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•  Representative  coverage  of  the  aptitude  areas  measured  by  the 
ASVAB  area  composites. 

•  High-priority  MOS  (as  rated  by  the  Army2  in  the  event  of  a 
national  emergency). 

•  Representation  of  the  Army's  designated  Career  Management  Fields 
(CMF). 

•  Representation  of  the  jobs  most  crucial  to  the  Army's  mission 
(e.g.,  the  combat  specialties). 

An  initial  set  of  19  MOS  representing  19  of  the  Army's  30  CMF  was 
chosen.  Of  the  11  CMF  not  represented,  two  (CMF  96  and  98)  are  classified, 
two  (CMF  33  and  74)  had  fewer  than  500  FY81  accessions,  and  seven  (CMR  23, 
28,  29,  79,  81,  84,  and  74)  had  fewer  than  300  FY81  accessions.  The  initial 
set  included  only  5  percent  of  Army  jobs  but  44  percent  of  the  soldiers 
recruited  in  FY81. 

Similarly,  of  the  15  percent  women  in  the  1981  cohort,  44  percent  are 
represented  in  the  sample;  of  the  27  percent  black.*,  44  percent  are 
represented  in  the  sample;  and  of  the  5  percent  Hispanic,  43  percent  are 
represented.  Although  female  and  minority  representation  is  high 
absolutely,  relatively  it  remains  about  the  same  as  in  the  population.  The 
sample  is  15  percent  female,  27  percent  black,  and  5  percent  Hispanic. 

Nine  of  the  19  MOS  were  earmarked  for  the  job-specific  performance 
measurement  phase  of  the  project.  These  were  selected,  as  a  subset,  with 
the  same  general  criteria  used  in  identifying  the  parent  list  of  19.  Since 
the  larger  list  is  composed  of  five  combat  and  14  noncombat  MOS,  it  seemed 
reasonable  that  these  categories  be  proportionally  represented  in  the  subset 


200CS0PS  (DAM0-0DM),  OF,  2  July  82,  Subject:  IRR  Training  Priorities. 
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of  nine.  Thus,  the  nine  MOS  designated  for  job-specific  performance 
measurement  development  were: 


(1) 

11B 

-  Infantryman 

(2) 

13B 

-  Cannon  Crewman 

(3) 

19E/K 

-  Armor  Crewman 

(4) 

31C 

-  Single  Channel  Radio  Operator 

(5) 

63B 

-  Vehicle  and  Generator  Mechanic 

(6) 

64C 

-  Motor  Transport  Operator 

(7) 

71L 

-  Administrative  Specialist 

(8) 

91A/B 

-  Medical  Care  Specialist 

(9) 

95B 

-  Military  Police 

These  nine  MOS  were  called  Batch  A  MOS;  the  remaining  ten  MOS  were  called 
Batch  Z  MOS. 

Refinements  of  the  MOS  sample  were  made  on  the  basis  of  two  cluster 
analyses  of  expert  ratings  of  MOS  similarity  and  reviews  of  the  sample  by 
the  Project  A  Governance  Advisory  Group  consisting  of  general  officers.  The 
final  sample  of  21  MOS  were  given  in  Table  1  (page  11). 


The  Battery  of  the  Predictor  Tests 


Analyses  of  the  concurrent  validation  data  indicated  that  65  reliable 
predictor  scores  could  be  obtained  from  the  6  spatial  tests,  10  computerized 
perceptual -psychomotor  tests,  and  the  temperament,  vocational  interest  and 
job  reward  preference  inventories  comprising  the  Trial  Battery  (Peterson,  et 
al.,  1987).  In  addition,  the  ASVAB  contains  9  subtest  scores  which  were 
obtained  from  Army  records.  These  74  scores  could  not  be  used  directly  in 
the  Project  A  concurrent  validity  analyses,  which  were  conducted  for  the 
most  part  on  an  MOS  by  MOS  basis.  First,  many  of  the  scores  were  highly 
intercorrelated  within  several  subsets  of  scores.  Second,  the  small  number 
of  cases  in  some  MOS  prohibited  (adhering  to  the  rule  of  at  least  10 
subjects  for  every  variable)  the  use  of  large  numbers  of  potential 
predictors  in  multiple  regression  equations  predicting  the  performance 
criteria.  The  74  predictor  scores  were  therefore  combined  into  20  predictor 
composites  before  the  predictor/criterion  relationships  were  explored. 
These  composite  scores  and  the  component  scores  or  scales  are  given  in 
Table  2. 

The  analyses  of  these  data  and  the  data  now  being  collected  from  the 
longitudinal  validation  sample  is  continuing.  There  may  well  be  changes  in 
the  composites  or  component  scales  as  a  result  of  later  analyses.  Before 
using  these  composites  or  component  measures  in  AFRP  analyses,  the  most  up- 
to-date  results  should  be  obtained  from  the  Project  A  researchers. 

The  Performance  Measures 

An  extensive  developmental  effort  has  been  undertaken  by  Project  A  to 
construct  a  variety  of  measures  for  use  as  performance  criteria  against 
which  to  validate  the  ASVAB  and  the  Trial  and  Experimental  Batteries.  In 
general,  two  type  of  measures  were  developed:  (1)  those  that  tested  ability 
to  perform  specific  MOS  technical  tasks,  and  (2)  those  that  assessed  more 
general  Army-wide  capabilities  (s.g.,  leadership  and  physical  fitness).  The 
performance  measures  that  will  be  administered  by  Project  A  in  1988  to  the 
first  tour  longitudinal  validation  soldiers  will  for  the  most  part  be  the 
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I^ble  2.  Predictor  Constructs  Emerging  From  Project  A  Analyses 
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Table  2.  Predictor  Constructs  Emerging  From  Project  A  Analyses  (continued) 
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same  as  those  used  in  the  earlier  concurrent  validation.  The  measures  will 
be  reviewed  for  currency  by  experienced  Army  officers  before  they  are 
administered.  The  Batch  A  MOS  measures  take  about  eight  hours  to 
administer;  the  Batch  Z  MOS  measures,  about  four  hours.  The  measures  are: 

1.  Hands-on  Tests.  Includes  both  general  soldiering  or  common  and 
MOS-specific  technical  tasks.  Approximately  15  tests  per  MOS. 
These  tasks  are  also  tested  in  a  written  mode  (see  number  2  below). 
(Batch  A  only) 

2.  Job  Knowledge  Tests.  Written  tests  covering  approximately  30 
technical  tasks,  including  both  conation  and  MOS-specific  activities. 
Tests  consist  of  approximately  250  multiple-choice  items.  (Batch  A 
only) 

3.  School  Knowledge  Tests.  Written  tests  covering  material  taught  in 
Basic  and  MOS-specific  training.  The  tests  contain  130-180 
multiple-choice  items  each.  (Batch  Z,  maybe  Batch  A) 

4.  Administrative  Measures.  This  information  is  available  from  Army 
files,  although  most  of  it  will  be  obtained  through  self-report. 
Includes  disciplinary  actions,  letters  of  appreciation,  awards  and 
badges,  SQT  scores,  weapons  qualification,  physical  readiness 
scores,  and  promotion  rate. 

5.  Armv-wide  Rating  Scales.  These  are  Behavioral  Summary  Scales  that 
will  be  completed  by  2-3  supervisors  and  (probably)  2-4  peers. 
There  are  10  dimensions  as  well  as  3  single  item  scales. 

6.  MOS- specific  Rating  Scales.  These  Behavioral  Summary  Scales  will 
be  used  by  2-3  supervisors  and  (probably)  2-4  peers  to  rate  the 
MOS-specific  performance  of  soldiers.  There  are  6  to  12 
dimensions,  depending  upon  MOS.  (Batch  A) 

7.  Combat  Performance  Prediction  Scale.  A  40- item  summated  rating 
scale  to  be  completed  by  2-3  supervisors  and  2-4  peers.  Raters 
assess  the  probability  of  individuals  reacting  as  the  soldiers  in 
the  examples  did  in  given  military  situations. 

8.  Job  History  Questionnaire.  Soldier  self-report  of  recency  and 
frequency  of  performance  on  the  30  technical  tasks  being  tested  in 
his  or  her  MOS.  This  may  be  revised  to  try  to  Identify  where ,  on  a 
hypothetical  learning  curve,  the  examinee  falls  on  several  (or  all) 
of  the  technical  tasks  being  tested  in  his  or  her  MOS. 

9.  Task  Performance  Rating  Scales.  Single  item  7-point  rating  scales 
that  cover  each  of  the  approximately  15  technical  tasks  per  MOS 
that  will  be  tested  hands-on.  (The  decision  to  use  this  for  Batch 
A  is  pending.) 
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10.  Common  Task  Rating  Scales.  A  total  of  117-point  scales  (Batch  Z). 
The  decision  to  include  these  scales  is  pending. 

11.  Job  Satisfaction  Questionnaire.  This  questionnaire  is  currently 
under  development. 


Generally  speaking,  the  same  types  of  measures  will  be  used  for  the 
second-tour  soldiers  from  the  concurrent  validation  sample  as  for  first- tour 
soldiers.  Modifications  and  additions  are  noted  below: 


1.  Hands-on  Tests.  Approximately  15  second-tour  tasks  for  each  MOS 
are  being  tested  in  the  hands-on  mode.  The  tasks  being  tested 
cover  primarily  skill  level  one  and  skill  level  two  activities. 
Both  common  and  MOS-specific  responsibilities  are  included.  These 
tasks  are  also  tested  in  a  written  mode  (see  number  2  below). 

2.  Job  Knowledge  Tests.  Approximately  30  second-tour  tasks  for  each 
MOS  are  being  tested  in  the  written  mode. 

3.  Administrative  Measures.  Promotion  Board  information  will  be  added 
to  the  pool  of  information  to  be  gathered  from  the  soldiers. 

4.  Army-wide  and  MOS-soecific  Behavioral  Summary  Scales.  Will  be 
modified  to  reflect  changes  from  first  to  second  tour.  A  decision 
as  to  whether  peer  ratings  will  be  gathered  is  pending. 

5.  Situational  Judgment  Test.  This  test  is  currently  under 
development.  The  examinee  will  be  asked  to  respond  to  a  variety  of 
supervisory  situations.  The  situations  will  probably  be  presented 
in  a  written  mode  and  responses  will  probably  be  multiple-choice. 

6.  Supervisory  Role-olav  Exercises.  The  examinees  will  participate  in 
two  role-playing  exercises.  In  one  exercise,  they  will  be  asked  to 
counsel  a  subordinate  and  in  the  other  exercise,  they  will  train  a 
subordinate  to  perform  one  or  two  technical  tasks.  The  subordinate 
will  be  played  by  the  scorer.  These  exercises  and  their  scoring 
protocols  are  currently  under  development. 
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FAMILY  RESEARCH  PROJECT/PROJECT  A 

DATA  ANALYSIS  PLAN  I 


The  analysis  of  any  large,  complex  set  of  data  with  many  different 
types  of  variables,  samples,  and  purposes  is  to  a  certain  extent  an  art 
form;  no  two  data  analysts  would  perform  the  same  analyses  in  the  same  order 
and  reach  exactly  the  same  conclusions.  However,  several  guiding  principles 
would  probably  underlie  the  approaches  taken  by  the  various  analysts. 
First,  they  might  seek  to  reduce  the  number  of  variables  to  a  more 
manageable  number.  At  the  same  time,  they  might  seek  to  increase  the 

reliability  and  meaningfulness  of  the  retained  variables.  They  might  then 
test  a  series  of  hypotheses  concerning  how  individual  variables  interrelate 
and  note  whether  the  interrelationships  hold  across  different  types  of 
samples,  possibly  pooling  samples  as  a  result.  Then  they  might  hypothesize 
whole  sets  of  interrelationships  among  various  kinds  of  variables 
(constructs)  in  the  form  of  models  and  test  whether  the  models  adequately 
account  for  the  observed  variable  interrelationships.  As  each  step  in  the 
analytic  process  informs  previous  steps  as  well  as  subsequent  ones,  the 
analysts  may  well  go  back  and  repeat  earlier  analyses  in  light  of  later 
findings.  In  short,  the  analysts  might  try  to  discover  what  can  be  done 
with  the  data  before  deciding  what  will  be  done. 

The  data  now  being  collected  from  10,000  graduates  of  Army  Advanced 
Individual  Training  (AIT)  schools  for  the  Army  Family  Research  Program 
(AFRP)  and  Project  A  lend  themselves  well  to  the  iterative  sequence  outlined 
above.  Data  are  being  collected  at  14  schools  for  21  Military  Occupational 
Specialties  (MOS).  The  number  of  AIT  graduates  in  the  sample  is  expected  to 
range  from  60  to  1,610  in  the  various  MOS  (see  Table  1).  Four  basic  types 
of  data  are  being  collected  for  these  soldiers: 

(1)  Information  obtained  from  the  Enlisted  Master  File  (EMF),  including 

Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  scores. 

(2)  Scores  obtained  on  the  Project  A  Experimental  Battery  administered  to 

the  soldiers  soon  after  they  entered  the  Army. 
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(3)  AIT  performance  measures  (knowledge  tests  and  rating  scales) 
administered  by  Project  A  prior  to  graduation. 

(4)  Responses  to  an  AFRP  Survey  of  AIT  graduates  administered  at  the  same 
time  as  the  AIT  performance  measures. 

This  plan  covers  the  analyses  of  these  data.  A  second  plan,  covering 
the  analyses  of  Project  A/ AFRP  data  collected  later  in  FY88  ami  early  FY89, 
will  be  written  when  the  form  of  the  AFRP  Army  Active  Duty  Family  Survey  to 
be  administered  to  the  Project  A  longitudinal  validation  sample  is  finalized 
and  the  exact  nature  of  the  Project  A  performance  measures  and  data 
collection  procedures  are  known.  (See  Gantt  chart  of  Project  A/AFRP  Joint 
Activities,  Figure  2).  It  should  be  noted  that  the  differences  between  the 
current  analysis  plan  and  the  subsequent  plan  will  largely  be  a  matter  of 
degree,  with  some  important  exceptions.  The  first  two  types  of  data  listed 
above  will  be  the  same  for  both  samples.  The  Project  A  performance  measures 

a 

will,  however,  be  more  extensive,  covering  additional  aspects  of 
performance.  Likewise,  the  AFRP  questionnaire  will  be  larger  and  will 
include  items  in  a  number  of  areas  not  covered  in  the  survey  of  AIT 
graduates.  Although  the  size  of  the  current  and  later  Project  A/AFRP 
samples  will  be  about  the  same,  approximately  2,000  of  the  soldiers  in  the 
longitudinal  validation  sample  will  have  previously  taken  the  AIT 
questionnaire.  Thus,  some  of  the  analyses  will  be  able  to  relate  changes  in 
familial  and  other  relevant  variables  to  performance,  retention  and 
readiness  measures.  In  addition,  the  analysis  of  the  later  data  will  be 
informed  by  the  results  of  the  AIT  analyses. 

This  analysis  plan,  like  its  successor,  will  concentrate  first  on  the 
analysis  of  the  AFRP  survey  data.  The  predictor  and  performance  measure 
data  from  Project  A  will  not  be  analyzed  separately.  Instead,  taking 
advantage  of  the  intensive  analyses  conducted  by  Project  A  on  data  from 
their  concurrent  validation  sample,  the  constructs  identified  in  these 
analyses  will  be  related  to  constructs  and  key  variables  derived  from  the 
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Year  of  AFRP  Contract 


Activity  |  1  |  2  |  3  |  4  | 

Administer  Project  A  Predictors 
and  School  Criteria  - - 

Develop  and  Pretest  AIT  Survey 

Administer  AIT  Survey 

Edit  and  Analyze  School  Data  - 

Develop  and  Pretest  Active  Duty 

Family  Survey  - 

Administer  Project  A  Performance 
Measures  and  Family  Survey  - 

Edit  and  Analyze  Survey  and 

Performance  Data  - 

Write  and  Review  Reports  — -  - 


Figure  2.  Army  Family  Research  Program/Project  A  Gantt  Chart 


AFRP  survey  analyses  in  the  framework  of  one  or  more  models.  Specific 
hypotheses  concerning  the  strength  and  direction  of  the  interrelationships 
among  the  sets  of  measures  will  guide  all  these  analyses. 

The  planned  analyses  have  been  sequenced  into  25  steps.  The  first  20 
steps  involve  only  the  data  from  the  AIT  graduate  survey.  The  last  5  steps 
incorporate  the  Project  A  predictor  and  performance  drta  in  the  model (s) 
developed  from  the  questionnaire  data.  Figure  3  gives  an  overview  of  the 
planned  analyses. 
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Multivariate  Analysis  of  AIT  Graduate  Survey  Data 


Preliminary  analyses  of  the  AIT  Graduate  Survey  data  will  be  conducted 
at  the  same  time  the  data  are  being  edited. 3  In  the  process  of  checking  the 
various  data  fields,  cumulative  frequency  distributions  will  be  examined  and 
the  proportion  of  respondents  who  marked  the  response  alternatives  of  each 
item  will  be  calculated.  For  the  purposes  of  the  multivariate  analyses 
described  in  this  section,  we  are  assuming  that  the  data  have  been  edited 
and  that  any  items  with  insufficient  response  variance  have  been  discarded. 

The  analysis  of  the  AIT  Graduate  Survey  data  is  made  complex  by  the 
branching  instructions  in  the  survey  itself.  These  instructions  led  all 
respondents  to  answer  questions  1-24  and  33.  All  unmarried  respondents  in 
addition  answered  questions  25-27.  Unmarried  respondents  with  a 
"significant  other"  also  answered  questions  28-32,  while  unmarried 
respondents  having  custody  of  one  or  more  child  answered  questions  34-39. 
All  married  respondents  answered  questions  36-48.  Married  respondents  with 
one  or  more  child  also  answered  34  and  35/ 

The  first  step  in  the  analysis  will  be  to  determine  the  number  of 
respondents  falling  in  each  of  the  branching  categories  by  MOS  and  in  total. 
Our  suspicion  is  that  there  will  be  only  enough  data  to  run  analyses  by  MOS 
on  all  cases  (Q  1-24,  33),  all  single  respondents  (Q  25-27),  and  possibly 
all  single  respondents  with  significant  others  (Q  28-32)/  Most  likely, 
owing  to  the  soldiers'  comparative  youth  and  earliness  of  their  Army  career, 
there  will  be  too  few  cases  to  run  analyses  by  MOS  on  the  married 
respondents  (Q  36-48),  and  the  married  and  single  respondents  with  children 

3This  analysis  plan  does  not  cover  the  editing  of  the  survey  data.  A 
memorandum  describing  the  planned  editing  process  is  being  produced  under 
the  direction  of  Dr.  Barbara  Moser. 

4Data  edits  performed  on  the  survey  file  will  check  whether  the 
respondents  correctly  performed  the  indicated  branching. 

3As  later  analyses  involving  the  Project  A  performance  data  might  best 
be  accomplished  by  MOS,  the  sufficiency  of  the  AIT  survey  data  to  enter  into 
these  integrated  (across  data  sets)  analyses  becomes  important. 
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(Q  34,  35  and  34-39,  respectively).  Initially,  however,  all  analyses  of  the 
AIT  data  will  be  accomplished  through  pooling  the  data  across  all  MOS. 
Analyses  by  MOS  (where  feasible)  will  be  then  run  in  order  to  determine 
whether  significant  variations  in  response  rates,  variable  relationships, 
and  model  goodness-of-f it  occur  across  MOS.  Groupings  of  MOS  will  be 
accomplished  to  aid  in  the  interpretation  of  the  data  when  the  number  of 
cases  are  too  small  to  run  separate  MOS  analyses. 

Where  the  number  of  cases  involved  permit,  the  total  sample  will  be 
divided  into  validation  and  cross-validation  subsamples.  The  total  sample 
will  be  stratified  by  MOS  and  any  familial  status  variable(s)  that  allow 
sufficient  cases  (5  or  more)  in  practically  all  the  cells.  Approximately 
sixty  percent  of  the  cases  in  any  one  cell  will  be  randomly  assigned  to  the 
validation  sample,  the  other  forty  percent  to  the  cross  validation  sample. 

The  major  steps  in  the  planned  analysis  of  the  AIT  survey  data  are 
outlined  below.  The  reader  should  bear  in  mind,  however,  the  iterative 
nature  of  the  analysis.  In  practice,  therefore,  earlier  steps  may  be  rerun 
in  light  of  later  findings. 

(1)  Classify  respondents  within  each  MOS  and  across  MOS  in  the 
following  manner: 


Children 
Mo  Children 


Si: 

Sianificant  Other 

lgle 

No  Sianificant  Other 

Married 

Decide  which  groups  are  large  enough  to  allow  separate  MOS  analyses  and 
which  can  only  be  analyzed  through  pooling  data  across  MOS.  Also  decide 
which  groups  are  large  enough  to  use  for  cross  validation  analyses. 

(2)  Select  the  validation  and  cross  validation  samples  for  the  total 
group  following  the  procedures  described  above.  The  cross  validation  sample 
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will  not  be  used  in  steps  3  to  19  below  except  as  necessary  to  have  enough 
data  to  run  specific  analyses. 


(3)  Categorize  survey  questions  1-24  and  33  under  several  initial 


general  constructs  as  follows: 

General  Construct 

Reasons  for  enlistment 

Satisfaction  with  Army 

Satisfaction  with  training  experience 

Training  staff  actions 

Importance  of  Army  performance  goals 

Comparison  of  military  and  civilian  life 

Future  plans  in  the  Army 

Importance  of  personal  factors 

Readiness 

Attitude  toward  Army  support  services 
Attitude  toward  family  life 
Familial  status 


Ho.  of 


Question  No.  Variables 


1  16 

2,  3,  4,  9,  10,  1  11 

5,  7  7 

8  9 

12  10 

13  18 

14  1 

15  9 

16,  17,  18,  19,  20  8 

21,  22  13 

23  10 

24,  33  4 


The  number  of  variables  listed  above  is  the  sum  of  the  number  of  subitems  in 
the  indicated  questions  that  can  be  converted  into  separate  variables. 


(4)  Using  the  validation  subsample  selected  in  step  2,  subject  each 
set  of  seven  or  more  variables  subsumed  under  the  general  constructs  to  an 
iterative  principal  factors  analysis  with  quartimax  rotation  of  factors  with 
eigenvalues  1.0  or  larger.  Examine  the  unrotated  and  rotated  factor 
solutions,  the  percent  of  variance  explained  by  each  of  the  factors,  and  the 
first  order  intercorrelations  along  with  the  content  of  the  variables.  For 
each  set  of  variables,  determine  one  or  more  subsets  that  could  be  combined 
to  form  a  coherent  dimension  or  composite  score. 


(5)  Determine  the  coefficient  Alpha  reliabilities  of  the  composites 
formed  as  a  result  of  step  4.  Remove  from  the  composites  any  items  that 
detract  from  the  reliability  of  the  composite  scores. 

(6)  Add  the  single  item  variable  on  future  plans  in  the  military 

(Q  14),  intercorrelate  it  and  the  revised  composite  scores  and  any 
uncombined  single  item  variables,  and  factor  analyze  the  matrix  as  in 
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step  4.  Again  examine  the  intercorrelations,  unrotated  and  rotated  factor 
solutions  and  the  percent  variance  explained  along  with  the  content  of  the 
composites.  Also  examine  bivariate  frequency  plots  to  determine  whether  any 
of  the  key  variables  have  nonlinear  relationships.  Determine  whether  the 
single  item  or  composite  variables  should  be  combined  with  others,  split 
among  two  or  more  composites,  retained  as  is,  or  dropped.  Determine  also  If 
any  nonlinear  transformations  of  variable  values  are  in  order.  If 
necessary,  repeat  step  4  with  new  groupings  of  variables. 

(7)  Repeat  step  5  for  any  new  or  revised  set  of  items  forming  a 
composite. 

(8)  Repeat  step  6,  adding  the  following  two  dichotomous  familial 
variables  to  the  matrix: 

1.  Single/married  (single  ■  1;  married  ■  2). 

2.  Having  child  (having  1  or  more  children  *  1;  no  children  »  0). 
These  two  variables  wiTl  be  carried  along  in  the  analysis,  but  will  not 
enter  directly  into  the  factor  analysis.  If  necessary,  repeat  steps  4-8 
until  satisfactory  composites  are  obtained. 

(9)  Separate  out  the  married  soldiers  from  the  single  soldiers  forming 
two  groups.  Run  confirmatory  factor  analyses  on  the  two  separate  groups  to 
see  how  well  the  factor  structure  obtained  in  step  8  fits  the  two  data  sets. 
(No  difference  would  be  expected  with  the  single  soldier  data  set,  since  it 
comprises  the  majority  of  the  cases  which  with  steps  4-8  were  run). 


(10)  Categorize  survey  questions  36-48  under  four  initial  general 


constructs  as  follows: 

General  Construct 

Ouestion  No. 

No.  of 
Variables 

Spouse  readiness 

37,  38 

2 

Satisfaction  with  Army  family  life 

36,  39 

9 

Spouse  support  for  military  life 

45,  46 

2 

Family  wellness 

47,  48 

2 
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(11)  Obtain  the  correlations  between  the  above  three  pairs  of 
variables.®  Decide  on  the  basis  of  these  correlations  whether  to  combine 
the  variables  to  form  one  or  more  composites  or  to  keep  the  variables 
separate. 

(12)  Factor  analyze,  as  in  step  4,  the  nine  variables  listed  under  the 
satisfaction  with  Army  family  life  general  construct.  Determine,  as  in 
step  5,  coefficient  Alpha  for  any  composite(s)  formed  as  a  result  of  the 
factor  analysis.  Modify  composite(s)  as  necessary. 

(13)  Factor  analyze  the  matrix  of  intercorrelations  formed  by  the 
composite  (and  single)  variables  defined  in  steps  8,  11,  and  12. 

(14)  For  the  single  soldiers  who  have  a  significant  other  (Q  27)?, 
categorize  survey  questions  26  and  28-32  under  two  general  constructs  as 


follows: 

No.  of 

General  Construct 

Ouestion  No. 

Variables 

Strength  of  relationship 

26,  28,  31,  32 

4 

Significant  other  support  for  military  life 

29,  30 

2 

(15)  Obtain  the  intercorrelations  between  the  above  6  variables. 
Oecide  on  the  basis  of  these  correlations  whether  to  combine  the  variables 
to  form  one  or  more  composites  or  to  keep  the  variables  separate. 


(16)  Factor  analyze  the  matrix  of  interrelations  formed  by  the 
composite  (and  single)  variables  defined  in  steps  8,  15,  and  16. 


^Combine  the  validation  and  cross  validation  samples  if  necessary  to 
build  up  the  sample  size.  Otherwise,  run  the  analysis  outlined  in  steps  Il¬ 
ls  on  only  the  validation  sample  of  married  soldiers. 

^Combine  the  validation  and  cross  validation  samples  if  necessary  to 
build  up  the  sample  size.  Otherwise,  run  the  analysis  outlined  in  steps  15 
and  16  only  on  the  validation  sample  of  single  soldiers  with  significant 
others. 
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(17)  Review  the  findings  of  steps  9  through  16  and  make  any  apparently 
desirable  changes  in  the  composite  definitions  for  the  married  and  single 
soldiers.  At  this  point  the  definitions  and  composition  of  the  composites 
will  be  allowed  to  be  different  if  the  interrelationships  among  the 
variables  warrant.  (Although  this  is  not  a  step  to  be  taken  lightly,  we 
must  keep  open  the  possibility  that  different  models  comprised  of 
differently  defined  constructs  may  be  best  suited  to  capture  the  impact  of 
antecedent  variables  on  the  retention  and  readiness  of  married  versus  single 
soldiers).  If  necessary,  repeat  some  previous  steps  until  satisfied  with 
the  composites. 

(18)  Identify  family  factor  variables  that  are  related  empirically  to 
the  composite  variables  and  the  individual  item  variables  retained  in  step 
17.  The  family  factor  measures  that  have  not  entered  the  analysis 
heretofore  as  variables  per  se  are  given  below: 


Family  Factor 

Ouestion  No. 

No.  of 
Variables 

Married/remarried 

24 

1 

Divorced  or  widowed/never 

25 

1 

Marriage  plans 

26 

1 

Special  other/no 

27 

1 

Number  of  children 

33 

1 

Age  of  youngest  child 

34 

1 

Children  at  AIT  location 

35 

1 

Years  of  marriage 

40 

1 

Spouse  vocational  activities 

41 

8 

Spouse's  age 

42 

1 

Spouse's  education 

43 

1 

Spouse  at  AIT  location 

44 

1 

As  mentioned  earlier,  it  is  suspected  that  the  response  variance  of 
some  of  these  variables  will  be  too  low  for  the  items  to  be  of  much  use  in 
multivariate  analyses. 


The  intercorrelation  of  these  family  variables  with  the  retained 
variables  will  be  obtained.  Of  course,  the  number  of  cases  for  which  there 
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is  bivariate  data  will  be  different  for  various  variable  pairs,  but  careful 
examination  of  the  matrix  should  enable  identification  of  the  family  factor 
variables  showing  most  promise  of  being  related  to  the  other  variables  of 
interest  in  the  AIT  graduate  sample.  The  intercorrelation  among  the  family 

factor  variables  themselves,  as  well  as  their  mutual  correlation  with  other 
variables,  will  allow  identification  of  any  family  variables  that  are 
essentially  redundant  or  that  could  be  combined  with  other  variables  to 
increase  their  psychometric  quality  and  meaningfulness. 

(19)  Assemble  the  variables  identified  in  steps  17-18  in  one  or  more 
models  applicable  to  the  total  group  of  respondents  and/or  major  subgroups 
of  respondents.  The  model(s)  will  generally  follow  the  family  model  in  the 
Army  Family  Research  Program  first  year  research  plan  (to  the  extent  that 
the  latent  constructs  of  that  model  are  represented  in  the  AIT  data  set). 
LISREL  confirmatory  factor  analyses  will  be  conducted  first,  to  assure  that 
the  relationships  of  the  observed  measures  to  the  latent  constructs  is  as 
specified  in  the  model(s).  Then  the  relations  among  the  constructs  will  be 
assessed  through  LISREL  structural  equations.  Chi  square  goodness-of-f it 
tests  and  the  goodness-of-f  it  index  and  root  mean  square  residual  will  be 
used  to  evaluate  and  compare  the  alternate  models.  In  this  regard,  the 
various  parameter  estimates,  matrix  of  residuals,  and  first-order  derivations 
for  the  fixed  parameters  will  also  be  examined.  Some  model  iterations, 
setting  selected  path  coefficients  at  zero,  may  also  be  tried  to  see  if  the 
various  variance/covariance  matrixes  can  be  more  parsimoniously  accounted 
for. 


(20)  Select  one  or  more  models  and  repeat  step  19  using  the  cross 
validation  sample.  Select  the  final  model(s)  to  use  in  the  next  set  of 
analyses. 
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Analysis  of  AIT  Survey  Data  with  Project  A  and  EMF  Data 


Intensive  analyses  have  been  conducted  by  Project  A  staff  of  the 
experimental  battery  of  predictor  measures  that  were  administered  to  the 
Project  A  concurrent  validation  sample.  These  analyses  resulted  in  the 
identification  of  the  24  composites  making  up  the  6  predictor  constructs 
given  in  Table  2. 

Analyses  of  the  performance  measures  also  administered  to  the 
concurrent  validation  sample  resulted  in  the  identification  of  the  five 
performance  constructs  given  in  Table  3.  These  tables  also  list  the 
observed  variables  that  constitute  the  given  predictor  and  performance 
constructs.  We  do  not  plan  to  repeat  these  analyses  on  the  Project  A 
longitudinal  validation  sample.  Instead,  we  will  adopt  the  predictor 
constructs  as  given  and  match  the  performance  measures  administered  to  the 
AIT  graduates  with  the  already  identified  Project  A  performance  constructs. 

After  forming  the  indicated  Project  A  composites,  we  will  explore 
alternate  ways  of  incorporating  these  two  sets  of  constructs  in  the  model (s) 
selected  in  step  20  above  through  hypothesizing  the  direction,  strength,  and 
signs  of  paths  connecting  the  constructs  already  contained  in  the  model (s) 
with  the  predictor  and  performance  constructs.  Again  LISREl  will  be  used, 
but  this  time  on  an  MOS  by  MOS  basis,  since  predictor  score  distributions 
will  vary  markedly  by  MOS  (as  a  result  of  Army  selection  and  classification 
procedures)  and  the  performance  measure  scores  are  relative  to  the  soldiers 
in  the  specific  MOS. 

(21)  Form  predictor  composites  using  the  weighting  procedures 
established  by  Project  A.  Form  the  job  performance  constructs  by  combining 
the  standard  scores  (within  MOS)  of  the  AIT  measures: 
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Table  3.  Performance  Constructs  Emerging  From  Project  A  Analyses 
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3rable  adopted  from  tables  presented  by  Campbell,  McHenry,  and  Wise,  1987.  Ratings  and  Written  Test 
Method  factors  omitted. 


Job  Performance  Construct 


AIT  Measure 


Core  technical  proficiency 

General  soldiering  proficiency 
Effort  and  leadership 
Personal  discipline 
Physical  fitness  and  military  bearing 


Technical  knowledge  and  skill  scale 
and  MOS  specific  knowledge  test 
items 

General  soldiering  task  knowledge 
test  items 

Effort  and  leadership  potential 
scales 

Self-control  and  following 
regulations  and  orders  scales 

Military  appearance  and  physical 
fitness  scales 


(23)  Identify  the  constructs  in  the  model(s)  from  step  20  that  one 
would  theorize  would  be  impacted  directly  or  indirectly  by  individual 
differences  on  the  predictor  constructs  and  their  associated  composites. 
Similarly,  identify  the  antecedent  and  succeeding  constructs  that  impact  or 
are  impacted  by  AIT  performance. 

(22)  Pool  the  validation  and  cross  validation  data  and  separate  the 
cases  out  by  MOS.  Pool  the  data  from  similar  MOS  that  have  insufficient 
cases  to  run  analyses  on  separately.  (Similarity,  in  this  case,  is  defined 
in  terms  of  selection  standards  and  ASVAB  score  distributions,  and  content 
of  AIT  training  as  judged  by  the  knowledge  tests.)  Run  analyses  of  variance 
comparing  mean  values  of  the  composites  and  retained  single  items  across 
MOS.  Use  Duncan's  a  posteriori  contrast  test  to  group  MOS  into  sets  with 
non-significant  mean  differences.  For  select  key  variable  pairs  test  the 
null  hypotheses  that  the  within  groups  regression  line  slopes  are  the  same 
for  all  MOS  and  that  they  are  all  equal  to  zero.  Also  test  whether  a  single 
regression  line  fits  the  bivariate  data  for  all  MOS. 

(24)  Run  LISREL  confirmatory  analyses  applying  the  model (s)  identified 
in  step  20  to  the  data  from  each  MOS  or  MOS  pool.  For  each  MOS,  the 
model (s)  will  be  run  twice,  once  without  the  predictor  and  performance 
construct  linkages  hypothesized  in  step  22  and  once  with  the  linkage  in 
place.  The  answers  to  two  questions  will  be  of  prime  interest  here:  (1)  How 
much  is  the  explanatory  power  of  the  antecedent  variables  increased  by  the 
addition  of  the  predictor  and  performance  constructs  to  the  model;  and  (2) 
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How  well  does  the  model  which  was  in  part  developed  on  the  basis  of  pooled 
data  across  MOS  apply  to  individual  MOS. 

(25)  Hake  any  indicated  changes  in  the  model (s)  which  could  simplify, 
increase  general izability,  or,  if  necessary,  enhance  goodness-of-fit  through 
making  the  model  specific  to  given  MOS  or  groupings  of  MOS.  Iterate  until 
satisfied  that  the  model (s)  are  both  meaningful  and  fit  the  empirical  data 
well. 
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